A Probabilistic Temporal Logic with Frequency Operators 

and Its Model Checking 



Takashi Tomita Shigeki Hagihara Naoki Yonezaki 

Dept. of Computer Science, 
Graduate School of Information Science and Engineering, 
Tokyo Institute of Technology 
{tomita, hagihara, yonezaki}(§fmx . cs . titech. ac. jp 

Probabilistic Computation Tree Logic (PCTL) and Continuous Stochastic Logic (CSL) are often used 
to describe specifications of probabilistic properties for discrete time and continuous time, respec- 
tively. In PCTL and CSL, the possibility of executions satisfying some temporal properties can be 
quantitatively represented by the probabilistic extension of the path quantifiers in their basic Com- 
putation Tree Logic (CTL), however, path formulae of them are expressed via the same operators 
in CTL. For this reason, both of them cannot represent formulae with quantitative temporal proper- 
ties, such as those of the form "some properties hold to more than 80% of time points (in a certain 
bounded interval) on the path." In this paper, we introduce a new temporal operator which expressed 
the notion of frequency of events, and define probabilistic frequency temporal logic (PFTL) based 
on CTL*. As a result, we can easily represent the temporal properties of behavior in probabilistic 
systems. However, it is difficult to develop a model checker for the full PFTL, due to rich expres- 
siveness. Accordingly, we develop a model-checking algorithm for the CTL-Iike fragment of PFTL 
against finite-state Markov chains, and an approximate model-checking algorithm for the bounded 
Linear Temporal Logic (LTL) -like fragment of PFTL against countable-state Markov chains. 

1 Introduction 

To analyze probabilistic systems, probabilistic model checking is often used. In probabilistic model 
checking, the inputs are a probabilistic model and a probabilistic property described in a specification 
language, and the output is whether or not the model satisfies the property. Probabilistic Computation 
Tree Logic M\M (PCTL) and Continuous Stochastic Logic ||2l|3l[I3 (CSL) are often used to describe 
specifications of probabilistic properties. PCTL and CSL are probabilistic extensions of Computation 
Tree Logic |6| (CTL) for discrete-time and continuous-time, respectively. In PCTL and CSL, the prob- 
abilistic path quantifier P is introduced in place of the universal path quantifier A (for all paths, . . .) and 
the existential path quantifier E (there exists a path such that . . .). As a result, we can quantitatively 
represent the possibility of executions satisfying temporal properties of interest. However, PCTL and 
CSL can only describe path formulae with temporal operators of the form "some properties hold in the 
next state" via the next-operator X; of the form "some properties eventually hold" via the eventually- 
operator F (or O); of the form "some properties always hold" via the always-operator G (or □); and of 
the form "some properties hold at a certain time point and other properties hold until that point" via the 
until-operator U. Thus, "property cp holds to more than 80% of time points (in the interval [0, 10]) on 
the path" cannot be represented in PCTL or CSL. To capture similar quantitative properties of an above 
example, CSL additionally has the steady-state operator S ||3l[T3, and there are also extensions of PCTL 
and CSL with reward (or cost) structure ifTOl . Even though, the steady-state operator S can only capture 
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the expected steady-state probability of being states satisfying properties of interest, and PCTL/CSL with 
rewards can only express the properties of the expected value of cumulated reward associated with states 
or transitions. 

To capture temporal properties of this kind, it is necessary to employ the integral of the duration 
of states, as in Duration Calculus |[T4l (DC). In DC, the above property is explicitly described by 
Jo^(p{t)dt > 8. In this paper, we describe such properties using the concept of frequency and introduce 
probabilistic frequency temporal logic (PFTL) based on CTL* [6], for discrete-time/continuous-time. To 
this logic, we add the (conditional) frequency operator Q. Using the frequency operator Q, we describe 
the above path property by Q^o*8*P PFTL. PFTL has rich expressiveness, and hence it is difficult to 
develop a model checker for the full logic (see Section IDl. However, we develop a numerical model- 
checking algorithm for the CTL-like fragment of PFTL against finite-state Markov chains (MCs), and 
a statistical model-checking algorithm for the bounded Linear Temporal Logic L6J (LTL) -like fragment 
of PFTL against infinite-state MCs. The outline of the numerical algorithm for the CTL-like fragment 
is similar to that of PCTL and CSL ||8l|3l|T0l. We compute transient and steady-state probabilities and 
reachability via matrix operations. The difference is that our technique requires the number of states 
satisfying the formulae of interest to be counted in terms of frequency. On the other hand, the statistical 
algorithm is an approximate one, based on statistical inference, and hence there are errors (although the 
significance level can be set according to our needs). However, we anticipate that it will provide useful 
information in many cases. We estimate whether or not "an input MC satisfies an input formula" using 
the sequential probabihty ratio test [ 1 1 1 (SPRT), as in Ull for CSL. 

The remainder of this paper is organized as follows. In Section|2l we give the definitions of discrete- 
time/continuous-time MCs, and describe their probabilistic behavior. In Section [3l we define the syntax 
and semantics of PFTL and discuss the expressiveness of PFTL. In Section HI we present the numerical 
model-checking algorithm for the CTL-like fragment of PFTL against finite-state MCs, and the statistical 
model-checking algorithm for the bounded LTL-like fragment of PFTL against infinite-state MCs. Our 
conclusions are stated in Section [5] 

2 Markov chains 

In this section, we present the definitions of discrete-time/continuous-time MCs and describe their prob- 
abiUstic behavior. We fix a set AP of atomic propositions that expresses the properties of interest. 

Definition 1. A (labeled) discrete-time Markov chain (DTMC) & is a tuple {S,s,P,L) such that: S is a 
countable set of states; sGS is an initial state; P : 5^ — ?> [0, 1] is a transition probability matrix satisfying 
the condition that Y^s'es^i^^^') — 1 ^'^'^ > 0} is finite for all s; L : 5 — >■ 2^^ is a labeling 

function that assigns to each state the set of valid atomic propositions in the state. 

P{s,s') denotes the probability of a one-step transition from s to s'. An execution (or discrete-time 
path) of a DTMC ^ is represented by an infinite sequence of states (O = sqSi . . ., where V/.P(5,-,5';+i) > 
and Q.f is the set of all paths starting from state s in For a path co = sqSi . . ., we denote the i-th state 
Si by 0){i) and the i-th suffix SjSi^i ... by co'. Let C^{so . . .5„) be a cylinder set {(0 G ^^^|V/ < n.(o{i) = 
Si}, and let Lq^» be the smallest a-algebra containing all the cyhnder sets C^{so, ■ ■ ■ ,s„) in Q.f^. The 

probabihty measure Prf^ on the measurable space (Q-f^,!.^^ ) is uniquely defined as follows: 

Prf^{Cf^{so...Sn))=flP{si^uSi). 

i=l 
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Definition 2. A (labeled) continuous-time Markov chain ( CTMC) ^ is a tuple {S, s, Q, L) such that: S is a 
countable set of states; s(^Sis an initial state; Q: S'^ ^'Kis an infinitesimal generator matrix satisfying 
the condition that Y.s'eS\{<i} Q{^-,^') = ~Q{^-,^)' Q{^^^') ^ if ^ 7^ ^' <^nd {s'\Q{s,s') > 0} is finite for all 
is a labeling function that assigns to each state the set of valid atomic propositions in the 

state. 

Q{s,s') is the rate of a one-step transition from 5' to 5'' if s ^ s'. Otherwise, —Q{s,s) is the exit rate 
from s and the spent time in s is exponentially distributed with parameter —Q(s,s). An execution (or 
continuous-time path) of a CTMC 'tf is represented by an infinite alternating sequence (O = sotositi ... or 
a finite and non-empty sequence (0 = sqIq . . . Sn°°, where Si G S and ti € M>o (this value represents the time 
spent in Si) for all / > 0. Q.f is the set of all paths starting from state s of For a path (O = sotosyti . . . 
{Sn°°), we denote the i-th state si by co{i), the i-th spent time f, by time{(0,i) and the suffix . . . 

after time point t by (of , where / = min{/'| £y^Qfy > t} and t[ = T!j=o^j — A path (0 is called an infinite 
time-length path if ^JlQf/me(ft),/) = °° (therefore, an infinite number of transitions do not occur in any 
bounded intervals of ]R>o on the path). For an interval / in ]R>o, let Cfg{so,Io,si,Ii,- ■ ■ ,In-i,s„) be a 
continuous-time cyhnder set {(O G Q.f{sQ)\(o{i) = Si Atime{(0,i) G /,}, and let ILqv be the smallest a- 

algebra that contains all cyhnder sets Cf^{so,Io ) '^1 ) ^1 ) ■ ■ ■ ) ^n— 1 ) '^(1 ) in ^fo' The probability measure Prf^ 
on the measurable space (H'^jEj^^ ) is uniquely defined as follows: 

Prlicl{so,k,suh,- ■ ■ ,ln-usn)) = fl • / -Qisusd-e^^'-'^'dt. 

We assume that CTMCs in this paper are not explosive, that is, almost all paths of them are infinite 
time-length. 

In numerical computations for CTMCs, transient probabilities are tractable and the uniformization 
method is a standard technique for computing transient probabilities of CTMCs. 

Definition 3. For a CTMC ^ = {S, s, Q,L) such that sup{—Q{s, s) \s G S} is finite, a uniformized DTMC 
unif^(jif) is {S,s,l + Q/X,L), where X is a uniformization rate greater than or equal to ^\i^{ — Q{s,s)\s ^ 
5} and I is the unit matrix. 

If each transition time in unif ;i (^) is exponentially distributed with parameter A , the behavior of 
unif;L('^) is equivalent to that of in a sense. For a uniformized DTMC unif;L(^) = {S,s,P,L), the 
transient probability matrix 11^ {Ilf{s,s') is the probability of being in state s', k time-units after the 
current state s in 'tf) is computed as follows: 

oo 

nf = £ p{n;Xk)-P" where p{n;U) is the Poisson distribution e^^'^iU)" /n\. 

n=0 

In the numerical computation, this infinite sum can be truncated. The truncation points can be determined 
by Fox-Glynn algorithm [TJ, which gives an (A/:) -size upper bound. 

3 Probabilistic frequency temporal logic 

In this section, we define the syntax and semantics of PFTL for discrete-time/continuous-time. We 
discuss the expressiveness of PFTL in Section [3^ 
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3.1 Syntax and semantics 

Definition 4. Probabilistic Frequency Temporal Logic (PFTL) is defined as follows: 

state formula (p ::= a \ ^(p \ (pi A (p2 \ Pr^p[\j/] 

path formula xj/ ::= (p \ -^xj/ {Yi^WiI I Vi^^Wi \ OLqiWilVi) 

where a G AP, ~,ixiG {<, >, <, >}, p,q G [0, 1] and I is an interval of N for discrete time (or o/R>o/or 
continuous time). 

Intuitively speaking, P^p[i/A] means that the occurrence probability of paths starting from the given 
state and satisfying y obeys the bound ~ p; Xy means that the suffix after the next state on the path 
satisfies Xj/; xin^^Wi means that 1/^2 holds at a certain time point in the interval / on the path and Xj/i holds 
until that point is reached; Ql(iq{Yi\¥2) means that the conditional frequency of time points satisfying 
Yi under the condition Y2 m the interval / on the path obeys the bound ixi q. We allow the following 
abbreviations: 

<PiV<p2 = -^h(pi^^92) 
true = (p\/^(p 

(Pl^(p2 = ^<Pl V <jt)2 
F'y = trueUV 

gV = -^f'^y 

In the sequel, we often omit the time bound / if / = [0,°°) and denote a time bound {i\i ixi A:} by ixi A: and 
a time bound {j\j it / € /} by / =F /. 

We now describe the semantics for DTMCs. The frequency in a finite interval is simply defined 
as the ratio of the number of time points satisfying subformulae in the interval. For an unbounded Q 
formula, we write a semantics (called limit semantics) in terms of the limit superior and limit inferior 
of the global frequency on the path. In general, the occurrence frequency of states in a path may not 
converge. However, if the MC is finite, we can regard it as simply the limit of the global frequency of the 
path, because of the convergence property of the limit distribution of a finite-state MC. 

Definition 5. Let the DTMC ^ = {S,s,P,L). For a state s eS, a discrete-time path (O, a state formula <p 
and a path formula Y> the satisfaction relation \= is defined as follows: 



&,s 1= a 




a G L{s) 


2!,s ^ -19 




^,s^(p 


Si,s'^(piMp2 


<^ 


^,s \= (pi and ^,s\= (p2 


9,s^P^p[Y] 


<^ 


Prf{{coeaf\^,co\=Y})--P 


&,ca\=(p 


<^ 




S>,ca \= ^Y 


<^ 




9,co\=Yi^W2 


<^ 


9,a)\=Yi and 9,a)\=Y2 


^,0}\=Xy 




9,(0^ \=Y 






3ieL{9,a)' \= Y2 and\/j < i.9,(0^ \= Yi) 
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true 

|{/€/| 



,(0' ^ Yi^¥2}\ 



\{ieI\S^,(o'^Y2}\ 

\{ien<kni\^,(o'^Yi^¥2}\ 

limsup -^^-77 — =- ■ — , ^ . , ^-r^ c<i q 

. A{i £N<knl\&,0)' ^ Yi ^¥2}\ 

i^oo \{ien<kni\^,(o'^Y2}\ 



if {jG/|^,co'> v^2} = 0, 
if sup/ G N, 

if sup/ = oo and ixiG {<, <}, 
otherwise. 



We define a semantics for CTMCs as follows. 

Definition 6. Let the CTMC ^ = {S,s,Q,L). For a state s £ S, a continuous-time path (O with infinite 
time-length, a state formula (p and a path formula Y, the satisfaction relation \= is defined as follows: 



'^,s\=a 


44> 


a G L{s) 








"if ,.v = (p\ A (p2 




'ia,s\=(pi and ^,s \= (p2 


^,s^P^p[xif] 




Prf{{coea'f\^,(O^Y})-P 






<r,w(0) 








'^,(o\=Yi^¥2 




^,(o\=Yi and 'rf,(o\=Y2 




<^ 


timeio),0) G M>o and "if, co»'"^(«'0) ^ y 


^, ffl ^ ¥i^'¥2 




3t e I. {"if, a' \= xi/2 and W G (O,?)-"^, 0)'' \= V^i) 



true 

Iimsup/j;|f^](0)><^ 



mi 



if{tel\'tf,(a'^xif2} = <D, 
if sup/ G M, 

if sup/ = oo and txiG {<, <}, 
otherwise. 



where fl^^\xj/2){t) is the frequency of time points satisfying Xj/i under \\f2 in the interval I-\-t, for the 
Lebesgue measure given by: 



n 



(W1W2) 



it) 



( \{t' el + tl"^, CO'' ^Yi^¥2}\ 
\{t'el-ht\^,(o''^xi/2}\ 

J^({?G/ + ?|^,aj^' H y/iAya}) 

^{{t' el+t\'^,a}'' \=xi/2}) 
undefined 



if sup/ ^ 00 and {t' G / + f 1*^, (o'' |= 1/^2} 7^ and 

^{{t' el + t\'^,(o'' ^xi/2})=0, 
if sup/ 7^ 00 and ^{{t' el + t\'^, a'' ^ ¥2}) > 0, 
otherwise. 



In continuous time, we must consider two cases: the number of time points satisfying subformulae 
in a finite interval is either only finite or continuously infinite. For finite time points, the frequency is 
defined in a manner similar to the discrete time. For continuously infinite time points, the frequency 
is defined as the ratio of the Lebesgue measure of the set of time points satisfying subformulae. By the 
following proposition (the proof is omitted from this paper), the set of time points satisfying subformulae 
is Lebesgue measurable. It is not necessary to consider the case in which there exists a countably infinite 
number of time points satisfying subformulae in a finite interval. 
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Proposition 7. For a CTMC 'la, a path formula Y, a bound interval I and a continuous-time path CO with 
infinite time-length, the set {t G /|^, (o' \= I/a} of time points satisfying Xj/ in I for CO can be expressed as 
a finite union of intervals. 

Note . For an unbounded formula (sup/ = 0°), we can define alternative semantics as follows: 

The above semantics (called stable semantics) would be tractable for analysis using automata-based 
methods, because it is captured by the co-Biichi condition. However, in the present paper, we use the 
limit semantics because it facilitates numerical model-checking. 



3.2 Expressiveness 

PFTL can flexibly express properties of paths via the frequency operator Q. We present some examples 
and note the expressiveness of PFTL. 

• Q.>qW- the global frequency of time points satisfying 1// on a path is greater than 0. 

- This formula is not equivalent to GFi/a representing is satisfied infinitely often on the 
path," because the global frequency on the path may converge to even if \\f is satisfied 
infinitely often. 

• Q>0 8^ ~ niors than 80% of the time points in [0,20] satisfy the proposition x = 10. 

- For probabiUstic systems, states are often associated with numerical values as in MCs with 

rewards. This formula is different than both Gt'^'^^'x = 10 and G'^-^^IS < x < 12. To capture 
behavior of a probabilistic system, we can write flexible expressions in PFTL. 

• P=i [Qm^<p]: the global frequency of the time points satisfying 9 obeys the bound to q for almost 
all paths. 

- This formula is equivalent to the CSL formula S[xi^[<p] if the given MC is irreducible (that 
is, it is possible to reach any state from any state). Otherwise, the S formula means that the 
expected value of the global frequency obeys the bound 1x1 q. 

• Q>o.9(V^|<P): rnore than 90% of time points satisfying <p on the path also satisfy Xj/. 

- If we assume probabilistic fairness, this formula is similar to a path formula G{(p — ;>P>o 9[i/7]) 
that means the probabilistic branching property P>o.9[v'^] holds at all states satisfying (p on the 
path. Furthermore, a conditional frequency (in a sense, it can be interpreted as a conditional 
probability) between path formulae on a path can be expressed via the Q operator without 
path quantifications. 

• -'Q>o.i<P A -iQ<o.9<p: the frequency of time points satisfying <p becomes less than 0.1 and also 

greater than 0.9 infinitely often. 

- Roughly speaking, this formula describes a situation in which intervals where <jp frequently 
holds and intervals where (p frequently does not hold appear alternately and become pro- 
gressively longer in both the limit semantics and the stable semantics. However, it is not a 
property of the languages defined by ffl-Kleene closure, e.g., ffl-regular and co-context free 
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languages. In the discrete-time stable semantics, for natural numbers qi and q2 such that 
< q\ < qi, a. single frequency formula Q-^y^/q^i^ilPi) is a property of ft)-context free. 
The class of G)-context free language is equivalent to the class of language accepted by co- 
pushdown automata lH, and we can construct an a)-pushdown automaton which stores the 
value n - {q2 — qi) —m- qi in the stack, where n and m are the numbers of visiting states sat- 
isfying (pi A (p2 and A (p2, respectively. Then Q^q^iq^ {(pi\(p2) can be represented by the 
automaton with the co-Biichi condition "the stored value n - {q2 — qi) —m- q\ is non-positive 
at finitely many time-points." 

4 Model checking 

In this section, we introduce model-checking algorithms. The inputs are a DTMC & = {S,s,P,L) (or 
a CTMC ^ = {S,s,Q,L)) and a formula (p. The output is whether or not \= (p (or "^,5 \= (p). 
Unfortunately, it is difficult to develop a model-checking algorithm for PFTL because of its high expres- 
siveness of path formulae, which describes linear time properties. In the model checking of linear time 
logic against a (non-) probabilistic system, an automata-based approach is generally used. In this type 
of approach, a (non-) deterministic cu-automaton equivalent to (the negation of) the input path formula 
Y is first constructed. Then the synchronized product system of the input system and the constructed 
w-automata is analyzed. Because the synchronized product system captures the intersection of the be- 
havior of the input system and that (out) of we can reduce the model checking to the reachability 
(or emptiness) problem. However, the language class of the path formulae in PFTL and its equivalent 
automata class are open in both the limit semantics and the stable semantics. The limit semantics does 
not primarily match existing automata, which do not have an concept of convergence. 

The stable semantics also results in intractable problems. For discrete time, the language class of 
the path formulae in PFTL is at least a superclass of ft)-regular, and includes G)-context free and non- 
w-regular languages and also non- ft)-Kleene closure languages. Hence, for model checking using an 
automata-based approach, we require a new type of automata to capture frequency. Such automata must 
have stack-like features, because they must be able to recognize some ^-context free languages. For 
continuous time, the set of the path formulae in PFTL is a superset of Metric Temporal Logic [9] (MTL), 
which is a real-time extension of LTL, in an interval-based semantics. Timed automata [1] are widely 
used as real-time automata, however, there exist MTL formulae (including bounded formulae lH) for 
which there is no equivalent timed automata. We conjecture that the required automata to satisfy some 
frequency conditions in continuous time is some kind of extended timed automata and that it is also 
impossible to construct such a timed automaton to capture a property represented by a path formula 
in PFTL. It may be possible to obtain a synchronized product directly, or it may not be necessary to 
employ an automata-based approach, but there is currently no available method for model checking of 
an LTL-like fragment of PFTL. 

Accordingly, we develop separate model-checking algorithms for two fragments of PFTL. The first 
is a strict numerical model-checking procedure for the CTL-like fragment of PFTL against finite-state 
MCs (Section im . The second is a statistics-based approximation model-checking for the bounded LTL- 
like fragment of PFTL against infinite-state MCs (Section 14.21) . The model checking for the bounded 
LTL-like fragment of PFTL against infinite-state DTMCs can be reduced to the model checking for 
LTL against finite-states DTMCs. Because, the number of reachable states from the initial state for 
bounded steps is finite and a bounded formula can be translated into a nested X formula. However, 
the translated formula has ^(inf/ + 2l^l)-size and hence it is difficult to check exactly for the bounded 
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LTL-like fragment of PFTL in the viewpoint of complexity. In a statistics-based approach, we sample 
prefix sequences of paths of an input MC by probabilistic simulation and statistically determine whether 
or not an input model satisfies an input formula by using the sample. Thus, we can apply statistical 
methods to model checking for an infinite-state MC, because it is easy to generate prefix sequences of 
paths of an MC even if the MC has infinitely many states. For finite-state MCs, numerical techniques are 
often limited by the state explosion problem. Statistical methods can also overcome also this issue. 

4.1 Model checking of the CTL-Uke fragment of PFTL 

In this section, we introduce a model checking algorithm for the CTL-like fragment of PFTL: 

state formula cp ::= a | -■<?) | <pi A (p2 | P~p[v] 
path formula y '■■= X<p | (piV'cpj \ Qi<ig{(p\\(p2) 

against finite-state MCs. 

The outline of the algorithm is similar to that for PCTL/CSL |l8l[3l[T0]|. We recursively compute a 
set Sat{(p) of states satisfying (p from sets of states satisfying subformulae of (p. 

Sat{a) = {s ^ S\s G L{a)} 

Sat{-^(p) = S\Sat{(p) 

Sat{(piA(p2) = Sat{(pi)r\Sat{(p2) 

Sat{P^p[Y]) = {seS\Prob^^'^{Y){s)-^p} 

where Prob^^'^ (y) is the vector of occurrence probabilities of paths satisfying y for c^ch starting state 
in discrete-time/continuous-time. 

In this paper, we indicate only how to compute Prob^^'^ (y) for the case Y = OL^qi^il^) ■ For y = 
Xq) or Y = <PiU^<P2> we can use procedure for PCTL/CSL. We assume that Sat{(p\) and Sat{(p2) are al- 
ready computed, and that an interval / is either of the form [k, k'] {k' ^ oo) or [^,0°), because all intervals of 
N can be represented in one of these forms, andProZ?''^(Q^g((pi [(pz)) is equal to Prob'^' {ij^^q'^'''^^^ (<Pi |<P2)) 
for / such that inf / 7^ sup/. 

P~p[Qm^(9i I92)] for DTMCs. If / = [k,k'] {k' £ N), we compute the occurrence probabihty of a path 
by counting the number of states satisfying (p2 and (pi A (p2 in the interval [k, k'] on the path. Let the vector 
j{s) be the occurrence probability of a path starting from s, visiting states in Sat{(p2) i times and states 
in Sat{(pi) nSat{(p2) j times, within h steps: 

'^1 if (/ = 0,7 = and s Sat{(p2)) or 

(/ = 1,7 = and s G Sat{(p2)\Sat{(pi)) or 
(/ = 1,7 = 1 and s G Sat{(pi) nSat{(p2)), 
otherwise. 

P{s, -) • vjz} if 5 G Sat{^i)nSat{q)2), 

P{s, — ) • v^y' otherwise, 
where P{n, — ) is the n-th row vector of the transition probability matrix P. 
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Here Pwb^ {Q'^^q\(pi\(p2)) is the probability of satisfying Q^^ *^^((pi|(p2) after ^-steps, and the k- 

step transition probability matrix is computed by P'^. Hence, we can compute Prob^ {Q^^\(pi\(p2)) as 
follows: 



Prob^iQ^^^^{(p,\cp2))=P' 



k'-k+\ 

■ I I 

i=0 j<i 



If / = [k,oo^ (unbounded), the basic idea is similar to the algorithm for the S operator in CSL ll3l[T0]|. 
Each path in a finite-state MC has to reach one bottom strongly connected component (BSCC) B (B 
is a strongly connected component, and s ^ B cannot reach s' B). BSCCs are computed by Tarjan's 
Algorithm (^(l^l)). For a non-BSCC A and BSCCs Bi, . . . ,B„, let the matrix P be reordered as 



Pa PaBi 
Pfi. 











Pab„ 






Pb,. 



where Pxy is a partial transition matrix from X to F of the transition probability matrix P with X , F C 5 
(we denote a partial transition matrix Pxx by Px)- 

For each path reaching the BSCC S,-, the occurrence frequency of states converges to the limit distri- 
bution Kb^ depending on B,. tt^, can be computed as the unique solution of the system of linear equations: 

KbiPb, = TiBi and Trgl = 1 

where 1 is the vector in which all elements are 1 . 

If the BSCC Bi has s € Sat{(p2), the global frequency of (pi under (p2 converges according to the limit 
distribution ;rg.. Otherwise, the global frequency is determined by the local frequency before reaching 
Bi. Then we compute the probability vector rA,rBi, ■ ■ ■ ,rB„ of reaching BSCCs for which the global 
frequency of (pi under (p2 obeys the bound M q. For the BSCC B,- having state s € Sat{(p2), 



fB, 



1 if B i n Sat {q>2)^(l) and 



^seBinSat{ipi)nSat(lf>2) 



^B.i^) 



llseBinSat{(p2] ^Bii^) 



txi q, 



otherwise. 

For the non-BSCC A, can then be computed as the unique solution of the system of linear equations: 

{PA-I)rA = - £ PAB.rBr 

0<i<n 

Finally, we compute the probability of reaching BSCCs having no state s € Sat{(p2) and satisfying the 
bound ixi ^. In a manner similar to the procedure used for v'- we compute the occurrence probability 
of a path by counting the number of states satisfying Y2 and Yi A Y2 until reaching BSCCs that have no 
state s € Sat{(p2) on the path. Let the vector u'j ^{s) be the occurrence probability of a path starting from 
s, visiting states in Sat{(p2) i times, states in Sat {(pi) Ci Sat {(p2) j times, and states in [jBjnSat{(fh)='!i^' ^ 



88 



A Probabilistic Temporal Logic with Frequency Operators and Its Model Checking 



steps the first time, within h steps. 



I Use U Bi, 

BinSat{lfh)=ld 

otherwise. 



if 5 A, 

if 5 G Sat{(pi)r\Sat{(p2) HA, 
if sG {Sat{(p2)\Sat{(pi))r\A, 
otherwise. 



The reason My -(i) = if 5 ^ A for > is that s cannot reach BSCCs having no state s G Sat{(p2) in 
h steps the first time. 

The probability of reaching BSCCs having no state s G Sat{(p2) and satisfying the bound ixi q can be 
obtained analytically as the infinite sum of i/j ^{s) for /j = to o°, because the number of steps required to 
reach BSCCs from states in the non-BSCC A is unbounded. However, we can adequately approximate the 
true probability for large h (see Section l4.3.1l) . Thus we can compute Prob^ {Q^q^ ((jPi |^)) as follows: 



h+l 



Prob^{Qt;^\cp,\cp2))=P'-{[riy,^,...,rl] +LL I ' 

h=0 i=0 j<i 

i>0^ jtxii-q 

where the superscript ^ means transposition of a vector. 



P^p[Qi^{(pi\(p2)] forCTMCs. On a uniformized DTMC unif;i (^) = (5, 5, P,L) of the input CTMC 
the occurrence probability of sequences sqSi ... of states can be captured by the techniques for DTMCs. 
Therefore, the remainder is the occurrence probability of sequences fo^i • • • of spent times such that the 
ratio of the total spent time in states obeys the bound ixi ^ on the path, for the uniformization rate A . 

Consider a simple case that / states {sq,... ,5,, / — 1 transitions) are in [0,^], the number of transitions 
is I (j <l < i) in [0, qk] and / — Z — 1 in the rest of the interval (qk, k] on the path. In this case, the total of 
to to is less than q ■ k and the occurrence probability of a sequence of spent times tQ... ti is 



p{l;Xqk) ■p{i-l- 1;A(1 -q)k) 



-Xqk (^#) ^-X{\-q)k (A(l-g)/:) 
/! ■ f/-/-l)! 



p(/- \;Xk) 



(/-l)!-g'-(l-^)'-'-i 



/!•(/-/-!)! 

As above, the occurrence probability of a sequence of spent times obeying the given frequency bound 
depends on only the numbers of states satisfying subformulae in the interval of interest, and it can be 
computed using the binomial distribution, because each spent time is independent and exponentially 
distributed with parameter A, and the Poisson probability p{i— \ ;Xk) is the occurrence probability of 
/ — 1 transitions in [0, k]. Under the other conditions, we can obtain similar results. Hence, the conditional 
probability Bt^q{j, i) of satisfying the frequency bound ixi q, when the numbers of states satisfying (p2 and 
<Pi A (p2 in the interval / are / and j respectively, is given by: 



n 

(-1 

I 

l=j 

I 

/=0 





{i-l)\.q^.{l-q)M 



if / = or (/ = i and 1 ixi ^) or ( j = and ixi ^) , 
if < 7 < /,0 < ^ < 1 and ixiG {<, <}, 

if < 7 < /,0 < <7 < 1 and ixiG {>,>}, 
otherwise. 
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Here Prob^{Q^q ' (91 192)) is the probability of satisfying ^ ^ |<P2) after k time units, analogous 
to the DTMC case, and the transient probability for k time units is H'f. Therefore, for a bounded interval 

I=[k,k'], 



' nf • (v« • Bm, (0, 0) + v« 1 • B^, (0, 1 ) + V? 1 • 5^, ( 1 , 1 ) ) if ^ = ^' , 

00 h+\ i 

nf • £ p(/i; A • (fc' - ^)) • £ £ v'lj ■ B^U, i) otherwise. 

h=0 i=0 j=Q 



In the numerical computation, this infinite sum for /j = to 00 can also be truncated as the computation 
of the transient probability Ylf. 

For an unbounded interval / = [k,oo), we can apply a routine similar to that used for DTMCs. The 
difference is that we must consider the cumulative binomial probability Bm^Ij, i) for u'j and the transient 
probability Ylf for k time units instead of P'^. 

Prob^{Q^^f{cp,\cp2))=nf-{[rlrl^,...y,f + ±'^^ 

h=0 i=0 j=Q 

4.2 Model checking the bounded LTL-Uke fragment of PFTL 

In this section, we introduce a statistical model-checking algorithm for infinite-state MCs and the bounded 
LTL-Uke fragment of PFTL: 

state formula 9 ::= Pr^p[Y] 

path formula Y ::= a\ \¥i^¥2\ Wi^'¥2 I Qm^(V^i l'/'2) 

where p £ (0,1) and / is a bounded interval of N for discrete time (or of M>o for continuous time). 

Because it is difficult to check exactly for a bounded LTL-like fragment formula in PFTL, we develop 
a statistics-based approximation model-checking algorithm. This techniques will provide us with useful 
information in many cases, even if it is not a strict model-checking procedure. In this approach, we 
sample finite prefix sequences of the paths of an input MC by probabilistic simulation and statistically 
determine whether or not the input MC satisfies an input formula by using the sample. We apply the 
sequential probability ratio test (SPRT) [11] to model checking, as was done in lfT2l for CSL. 

4.2.1 Sequential probability ratio test 

The SPRT is a sequential hypothesis test developed by Wald ||TT'|. In a sequential test, the sample size 
is not fixed: observations are sequentially generated until the sample data indicate which hypothesis to 
supported under predesigned conditions. In SPRT, we preset the type I error rate a > 0, the type II error 
rate jS > 0, and the indifference region width 25 > 0. For a formula P^p[i/A] {p±5 £ (0, 1)), we test the 
null hypothesis Hq: p> p + 5 against the alternative hypothesis Hi : p < p — 5, where p is the true value 
of the occurrence probability of paths satisfying 1//. If the hypothesis p = 6 is true, the number m of paths 
satisfying i//^for a sample size n is binomially distributed n\d'"{l — 6)"^'" / {m\{n — m)l). Conversely, this 
value represents the likelihood of the hypothesis p = 6 if we observe that m paths satisfy Y for a sample 
size n. Therefore, the likelihood ratio A of Hq to Hi for a sample {(0\ , . . . , a)„} is: 

A((»„.,...„}) + -<'' + ^»'" 
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where m = |{£0,-|£0,- |= V^}|. 

Here Hq is more likely than Hi for a given sample if the likelihood ratio is greater than 1 and H\ 
is more likely than Hq for the sample if the likelihood ratio is less than 1. For an observed sample 
{(Oi,...,(On} and error rates a and j3, the next action is determined as follows: 

' Accept //o if A({a)i,...,a)„}) > (l-j8)/a, 

< Accept //i if A({a)i,..., co„}) < j3/(l -a), 

Observe and add co„+i to the sample otherwise. 

As a result, the probability of accepting the hypothesis Hq is at least 1 — aif^>;7 + 5, and at most J3 if 
p < p — 5. If \p — p\ < 5, the hypotheses are indifferent at error rates a and j3. 

4.2.2 Satisfaction checking for bounded path formulae against paths 

To carry out a test, we must check w |= )/a for a sample path co and a bounded formula y with the total 
boundary ktotai- Whether or not (0 \= Xj/ does not depend on the suffix after ktotai steps/time-units of (O. 
For the finite prefix on [O,ktotai] of £0, we recursively compute an ordered set SatInt(o{^f) of subintervals 
satisfying i/a in [O,k,otai\, using ordered sets of subintervals satisfy subformulae of \\f. We can then derive 
(o\=\irii there exists / G SatInt(a{Y) such that € /. 

We assume that SatInt(o{^f\ ) and Satlntfoi^i) are already computed and merged. By writing SatlntaiY) 
= {/;,...,/„}, we mean that the set {/,-,...,/„} satisfies /, fl/j+i = 0, sup/; < inf/j+i and sup/, = inf/,_|_i 
sup/, ^ /;,/,+!. In this paper, we do not include an algorithm for DTMCs, because the structure of a 
discrete-time path is simple, and it is not worthwhile to pursue the matter. 

a G AP for CTMCs. For an atomic proposition a G AP, the set of intervals satisfying a is determined 
immediately by the labeling function L. Therefore, SatInto}{a) = {[Ly-^o^""^(®)7))E;=o^™^(®)y))l'^ ^ 
L(a)(0)}. 

-1 V^i for CTMCs. SatInt(o {~'Yi ) ^ of intervals complementary to the union of intervals in Satlnta){\l/i ) 
in [0, ktotai]- Therefore, for SatInta{Wi) = SatInt(o{^Vi) = {[0,inf/i] [sup/„,^«o<fl/] \4} U 

U-r/{([sup/i,inf/m]\//)\/m}. 

Yi A Yi for CTMCs. Satlntf^iYi A 1//2) is a set of intervals intersecting each element of SatInta,{Yi) 
and each element of Sat Into, {^2)- Therefore, for Satlntoj {Yi) = {h,---in} and Sat Into) {Y2) = {Ji:---Jm}, 
SatlntcoiVi A Y2) = ULi U7=iU- n/^}. 

xl/il]'\l/2 for CTMCs. Let SatIntco{y/i) = {h,- --In} and SatInt(a{\j/2) = {Ji,-- ■■Im}- For time points 
f G /, U {inf//} and t' G Jj such that t < t', there exists /' G SatInt(aiWi^'¥2) such that ? G /' if {t,t') C 
and G / + In this case, t' is in (/, Usup//) fl/j (= and t is in where infX,j- = infJ^j- — 
sup/, supX,;^- = supYij - inf/, (inf y,;^- G yij- A sup/ €l)4^ inf^i,i G and {supYij G i^-,; A inf/ el) 4^ 
sapXij G X,;^. In addition, SatIntco{xif2) C 5a?/n?to( 1/^111^1/^2) if G /. Therefore, 

m n 

5a?/n?«(vAiUV2) = U U{^U ^ (// U {inf/,})} U 



{Satlntai^l) if G /, 
otherwise. 
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OLgiWilWi) for CTMCs. If / = [0,0], Q^f (v^ilv^i) is just a conditional statement. Therefore, 

SatInt(o{Q^Mq'' (V^i I ^2)) is equal to SatInta,{Y2 — )• V^i ) if 1 cxi ^, SatIntco{Y2 ^¥1) otherwise. 

If MI >0,'^,co\= Qi^{Yi\¥2) ^ ^, co^" \= Q'^^" iWilWi) by Definitionil Therefore, if MI > 0, 
SatInta,{Ql,^{Yi\¥2)) = {J - mfI\J G SatInta,{Ql^^" {Yi\¥2))- 

If MI = and sup/ = ^ > 0, SatIntco{Qi<ci{¥i\¥2)) satisfy the property 7 E SatInta,{Qi^{¥\\W2)) 
J G {^1/(1^1 1 v/2>(^) txiq} for any interval/. Therefore, we determine Saf /nfcj (Q^^'"*^( V^i |v^2)) by analyzing 
/(Vi|v2>'-^)' ^i''^'^' f^'' SatInt(o{Yi A 1/^2) = and Sat Int (0(^2) = {-/i, • • •■/m}, we compute a set 

nondif(o(vAi |i//2) of candidates for non-differentiable points of f\^,^\^,^_){t) ■ 

nondif£o(vAi|v/2) = {0,/:foffl/ - ^} U {inf/' - ^,inf/',sup/' - ^,sup/'|/' G {/i, ... ... ,/,„}}. 

Let {?!,...,?/} be the ordered elements of nondifo((i/^i \ ¥2))- The truth values of 1//2 and A i//2 are 
unchanged in each interval and + because if their truth values did change, there would 

have to be other non-differentiable points between f,- and Thus f\^,^\^^^_){t) is monotonically increas- 
ing, monotonically decreasing, fixed, or undefined in the interval (f,-, ). In addition, /'^^ i^^) (0 i^ equal 

to4;;ii:r*(Ofor.G(.,-,r,-+o- 

Hence, for a non-differentiable time point ti, [ti^tj] G SatIntci){Ql^{Yi\Y2)) if f{\ffi\\j/2)(^') ^ ^- ^^'^ 
an interval between non-differentiable time points, we determine whether or not or a 

subinterval of it, is in 5'af/?i?o(Q^^(i/Ai ly/2)) as follows. For ^^{t) = ^{[Ji'eSatint{\i/)^' l~l (•^ + 0)' 

1. If ^^^^^\ti) = and ^l^^''\ti+x) = 0, 1/^2 and i/^i A 1/^2 do not hold on an interval C with 
positive time length. Therefore, if /^^jj^^^) ^~ -^('v/iti^i) ^^'^ ^^'^ ^' ^ ^' < undefined or obeys 
the bound txi q, then G SatInt(o{Qi^g{Yi\¥2))- 

2. If = and > 0, /f^,|^^)(0 is fixed and equal to in the interval 
{ti,ti+i). Therefore, if obeys the bound M then G 5af/?i?£o(Q^^(i/^i |i//2))- 

3. If ^!^/\ti) > and ^!^/\ti+i) = 0, is fixed and equal to ff^'i^^,j{ti) in the interval 
{ti,ti+i). Therefore, if obeys the bound M then G 5af/?ifa)(QMg('/^i I V^2))- 

4. If ^jj^l''^\ti) > and ^^^'^'(fi+i) > 0, /^J^ijl^^)'-^-' nionotonically increasing, monotonically de- 
creasing, or fixed in (f,-, ). Therefore, if both fj^'^^^^^-j (U) and /l^^^jl^^^ (^f'+i ) obey the bound cxi q, 
then G SatInto,iQi^g{\l/i\V2))- Moreover, if either /JJ;,!^^)!?/) or obeys the 
bound txiq, then or {t'{,ti^i) G SatIntto{Ql^{Yi\Y2)) where f- satisfies: 

with a = (if|°x'k(^'+i) -■^i?Ak(^')) -^0 and b = {^^'\u+,) - ^^'\tt)) /{U+,-ti). 

— f; H ; • 

<3 — 

In addition, because f'{t-) = q, [t'i,t-] G SatInta,iQi^y{Yi\V2)) if IXG {<,>}• 
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4.3 Complexity 

4.3.1 Complexity of model checking for the CTL-like fragment of PFTL 

For a DTMC & = {S,s,P,L) or a CTMC = {S,s,Q,L) (and its uniformized DTMC unif;t(^) = 
{S,s,P,L)) and a CTL-like fragment formula (p, the time complexity of model checking is linear in 
|<p|, which is the number of operators in (p, and polynomial in \S\, which is the complexity of the re- 
cursive procedure for each operators. Except for a Q formula, the time complexity is the same to that 
for PCTL/CSL [8, 3]. For each bounded P~p[Q^^(«?>i |«?>2)] (sup/ < oo), computing the sum of vectors 
v'j I takes ^(IS'P • k^) time, where k = sup/ for DTMCs or ^ = A • sup/ for CTMCs. For each un- 
bounded P~p[Q^g(9i |<P2)]> computing the limit distributions Tie, , . . . , tt^^^ and the reachability vectors 
rA,rB^, . . . ,rB„ takes time, where A and B\,...,B„ are a non-BSCC and BSCCs of P, respec- 

tively. If the input MC is reducible, an additional computation of the transient probability Tlf and the 
sum of vectors . takes • k' + • | loge|^^) time, where k' = inf / for DTMCs or ^' = A • inf/ for 

CTMCs, and e is the maximum absolute value of the eigenvalues of the partial matrix Pa consisting the 
non-BSCC A of P. This is because the probability vector of reaching BSCCs within loge|~^)-steps 
is sufficiently close to the probability vector of reaching BSCCs within an unbound number of steps. 

4.3.2 Complexity of model checking for the LTL-like fragment of PFTL 

The complexity of model checking for the LTL-like fragment of PFTL is divided into two parts, the 
complexity of the sample used in the testing and the complexity of the observations and satisfaction 
checking for a sample trace of path. Regarding the sample size, approximations for the expected sample 
size are provided in ifTTl [T3l . This size depends on the chosen significance level and the difference 
between the query value p and the true probability p^f, for an input formula P^p[i/A]. However, this is not 
specific to our method, and the details of the expected size are omitted from this paper. The observation 
of a sample path is just a probabilistic simulation, and its time complexity is ^{ktotai 'log \E\)lff{X - kfotai • 
logl^l) where kiotai is the total boundary of the input formula, l^l is the number of transition choices 
of the input MC and A is the average exit rate of the input CTMC, for the input DTMC/CTMC. For an 
input formula P^p[i/A] and a DTMC, we need only count states satisfying subformulae for each operator. 
Therefore, the satisfaction checking takes ^{ktotai 'IwD time, where is the size of Y- However, on 
a CTMC, the size of the set of intervals satisfying formulae is at worst twice that of the set of intervals 
satisfying subformulae, for each Q operator. Thus, the satisfaction checking takes ff{X -kfotai 'Iwl' 2^^^^) 
time, where \ y\q is the number of Q operators in Y- In practice, many intervals satisfying formulae are 
merged, because each spent time on a state is exponentially distributed with the exit rate of the state and 
the probability of generating a bad sample path by probabilistic simulation is negligible. 

5 Conclusions and future directions 

We introduced the frequency operator Q and defined the syntax and semantics of PFTL. PFTL has rich 
expressiveness, and it is difficult to develop a model checker for the full logic. However, we developed 
a numerical model-checking algorithm for the CTL-like fragment of PFTL against finite-state MCs, and 
a statistical model-checking algorithm for the bounded LTL-like fragment of PFTL against infinite-state 
MCs. The statistical model-checking is not strict, but we anticipate that it will provide useful information 
in many cases. Especially, it is worth noting that the Q operator can, in a sense, express a conditional 
probability between path formulae, without path quantifications. 
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Our extension is based on an intuitive idea for describing a property of a behavior, especially in a 
probabilistic system. Although, it is difficult to strictly check a model for the logic, and also the non- 
probabilistic version of PFTL, because it is intractable from the viewpoint of automata theory. Therefore, 
it will be necessary to find treatable and useful fragments of the logic and classes of restricted models. 
This is one future direction of our research. Another future direction is to provide approximate model- 
checking against more complex systems, or for further extended logics. In this paper, we have assumed 
that our model is an MC. Nevertheless, we can apply this type of approximate model-checking via 
statistical methods to more general stochastic processes, e.g., systems of stochastic ordinary differential 
equations (continuous states and continuous transitions), because we can directly use discretized traces 
of paths obtained from stochastic simulations. Also, it is not difficult to check whether or not a sample 
path satisfies a bounded property such as "92 holds in the interval [0, 10] and cpi holds to more than 90% 
of the time points until that point" (frequently (pi until (p2). 
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